Video decoder architecture and method for using same

ABSTRACT

A decoder and method for using a new picture or frame type is provided. This type is referred to a an SP-picture. The temporal redundancies are not exploited in I-frames, compression efficiency of I-frame coding is significantly lower than the predictive coding. A method allows use of motion compensated predictive coding to exploit temporal redundancy in the sequence while still allowing perfect reconstruction of the frame using different reference frames. Methods using this new picture type provide for error resilience/recovery, bandwidth scalability, bitstream switching, processing scalability, random access and other functions.  
     The SP-type picture provides for, among other functions, switching between different bitstreams, random access, fast forward and fast error-recovery by replacing I-pictures to increase the coding efficiency. As will be demonstrated, SP-pictures have the property that identical SP-frames may be obtained even when they are predicted using different reference frames.

CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application is a continuation of U.S. patent applicationSer. No. 09/827,796 filed on Apr. 6, 2001 entitled VIDEO DECODERARCHITECTURE AND METHOD FOR USING SAME and claims priority from same andProvisional Application No. 60/259,529 filed on Jan. 3, 2001, bothincorporated herein by reference.

BACKGROUND

[0002] This invention relates generally to the field of the multimediaapplications. More particularly, this invention relates to a new frametype, apparatus and method for using same to provide for access of avideo stream.

[0003] Multimedia applications that include audio and streaming videoinformation have come into greater use. Several multimedia groups haveestablished and proposed standards for compressing/encoding anddecompressing/decoding the audio and video information. The examples areMPEG standards, established by the Motion Picture Expert Group andstandards developed by ITU-Telecommunications Standardization.

[0004] The following are incorporated herein by reference:

[0005] G. Bjontegaard, “H.26L Test Model Long Term Number 6 (TML-6)draft0”, document VCEG-L45, ITU-T Video Coding Experts Group Meeting,Eibsee, Germany, 09-12 Jan. 2001. Keiichi Hibi, “Report of the Ad HocCommittee on H.26L Development”, document Q15-H-07, ITU-T Video CodingExperts Group (Question 15) Meeting, Berlin, 03-06 Aug. 1999. Gary S.Greenbaum, “Remarks on the H.26L Project: Streaming Video Requirementsfor Next Generation Video Compression Standards”, document Q15-G-11,ITU-T Video Coding Experts Group (Question 15) Meeting, Monterey, 16-19Feb. 1999. G. Bjontegaard, “Recommended Simulation Conditions forH.26L”, document Q15-1-62, ITU-T Video Coding Experts Group (Question15) Meeting, Red Bank, N.J., 19-22 Oct. 1999. ATM & MPEG-2 IntegratingDigital Video into Broadband Networks by Michael Orzessek and PeterSommer (Prentice Hall Upper Saddle River N.J.).

[0006] Video sequences, like ordinary motion pictures recorded on film,comprise a sequence of still images, and the illusion of motion iscreated by displaying consecutive images at a relatively fast rate. Forexample, the display rate are between five and thirty frames per second.Because of the relatively fast frame rate, the images in consecutiveframes tend to be similar. A typical scene recorded by a cameracomprises some stationary elements, such as, for example, backgroundscenery and some moving parts. The moving parts may take many differentforms, for example, the face of a news reader, moving traffic, and soon. Alternatively, the camera recording the scene may itself be moving,in which case all elements of the image have the same kind of motion. Inmany cases, this means that the overall change between one video frameand the next is rather small. Of course, this depends on the nature ofthe movement, the rate of the movement, i.e., the amount of change fromone frame to the next.

[0007] The purpose of the video coding is to remove the redundancy inthe image sequence so that the encoded data rate is commensurate withthe available bandwidth to transport the video sequence while keepingthe distortion between the original and reconstructed images as small aspossible. The redundancy in video sequences can be categorized intospatial and temporal redundancy. Spatial redundancy refers to thecorrelation between neighboring pixels in a frame while temporalredundancy refers to correlation between neighboring frames.

[0008]FIGS. 1A-1C illustrate the type of encoded/compressed video framesthat are commonly utilized for video standards. FIG. 1A depicts anIntra-frame or I-type frame 200. The I-type frame or picture is a frameof video data that is coded exploiting only the spatial correlation ofthe pixels within the frame without using information from the past orthe future and is utilized as the basis for decoding/decompression ofother type frames. FIG. 1B is a representation of a Predictive-frame orP-type frame 210. The P-type frame or picture is a frame that isencoded/compressed using motion compensated prediction from I-type orP-type frames of its past, in this case, I.sub.1 200. That is, previousframes are used to encode/compress a present given frame of video data.205 a represents the motion compensated prediction information to createa P-type frame 210. Since in a typical video sequence the adjacentframes in a sequence are highly correlated, higher compressionefficiencies are achieved when using P-frames.

[0009]FIG. 1C depicts a Bi-directional-frame or B-type frame 220. TheB-type frame or picture is a frame that is encoded/compressed using amotion compensated prediction derived from the I-type reference frame(200 in this example) or P-type reference frame in its past and theI-type reference frame or P-type reference frame (210 in this example)in its future or a combination of both. B-type frames are usuallyinserted between I-type frames or P-type frames. FIG. 2 represents agroup of pictures in what is called display order I.sub.1 B.sub.2B.sub.3 P.sub.4 B.sub.5 P.sub.6. FIG. 2 illustrates the B-type framesinserted between I-type and P-type frames and the direction which motioncompensation information flows.

[0010] A system for P-frame encoding and decoding is provided and isshown in FIGS. 3 and 4. Referring to FIGS. 3 and 4, a communicationsystem comprising an encoder 300 of FIG. 3 and a decoder 400 of FIG. 4is operable to communicate a multimedia sequence between a sequencegenerator and a sequence receiver. Other elements of the video sequencegenerator and receiver are not shown for the purposes of simplicity. Thecommunication path between sequence generator and receiver may takevarious forms, including but not limited to a radio-link.

[0011] Encoder 300 is shown in FIG. 3 coupled to receive video input online 301 in the form of a frame to be encoded I(x, y), called thecurrent frame. By (x, y) we denote location of the pixel within theframe. In the encoder the current frame I(x,y) is partitioned intorectangular regions of M×N pixels. These blocks are encoded using eitheronly spatial correlation (intra coded blocks) or both spatial andtemporal correlation (inter coded blocks). In what follows weconcentrate on inter blocks.

[0012] Each of inter coded blocks is predicted using motion informationfrom the previously coded and transmitted frame, called reference frameand denoted as R(x,y), which at given instant is available in the framememory 350 of the encoder 300. The motion information of the block maybe represented by two dimensional motion vector (Δx(x,y), Δy(x,y)) whereΔx(x,y) is the horizontal and Δy(x,y) is the vertical displacement,respectively of the pixel in location (x,y) between the current frameand the reference frame. The motion vectors (Δx( ), Δy( )) arecalculated by the motion estimation and coding block 370. The input tothe motion estimation and coding block 370 are current frame andreference frame. The motion information is provided to a MotionCompensated (MC) prediction block 360. The MC prediction block is alsocoupled to a frame memory 350 to receive the reference frame. In the MCblock 360, the motion vectors for each inter block together with thereference frame are used to construct prediction frame P(x, y):

P(x, y)=R(x+Δx(x,y), y+Δy(x,y)).

[0013] Notice that values of the prediction frame are calculated onlyfor inter blocks. For some pixels (x,y) which belong to intra blocksthese values will not be calculated. It is also possible to use morethan one reference frame. In such case different blocks may usedifferent reference frames.

[0014] Subsequently, the prediction error E(x, y), i.e., the differencebetween the current frame and the prediction frame P(x, y) is calculatedby:

E(x, y)=I(x, y)−P(x, y).

[0015] In transform block 310, the prediction error for each K×L blockis represented as weighted sum of a transform basis functionsf.sub.ij(x, y),${E\left( {x,y} \right)} = {\sum\limits_{i = 1}^{K}\quad {\sum\limits_{j = 1}^{L}{{c.{sub}.{{err}\left( {i,j} \right)}}{f.{sub}.i}\quad {{j\left( {x,y} \right)}.}}}}$

[0016] The weights c.sub.err(i,j), corresponding to the basis functionsare called prediction error coefficients. Coefficients c.sub.err(i,j)can be calculated by performing so called forward transform. Thesecoefficients are quantized in quantization block 320:

I.sub.err(i, j)=Q(c.sub.err(i,j),QP)

[0017] where I.sub.err(i, j) are the quantized coefficients. Theoperation of quantization introduces loss of information—the quantizedcoefficient can be represented with smaller number of bits. The level ofcompression (loss of information) is controlled by adjusting the valueof the quantization parameter (QP).

[0018] The quantization block 320 is coupled to both a multiplexer 380and an inverse quantization block 330 and in turn an inverse transformblock 340. Blocks 330 and 340 provide decoded prediction errorE.sub.c(x, y) which is added to the MC predicted frame P(x, y) by adder345. These values can be further normalized and filtered and the resultstored in frame memory 350.

[0019] Motion vectors and quantized coefficients are encoded usingVariable Length Codes (VLC) which further reduce the number of bitsneeded for their representation. Encoded motion vectors and quantizedcoefficients as well as other additional information needed to representeach coded frame of the image sequence constitute a bitstream 415 whichis transmitted to the decoder 400 of FIG. 4. Bitstream may bemultiplexed 380 before transmission.

[0020] The special type of the inter coded blocks are copy coded blocks.For copy coded blocks values of both motion vectors and quantizedprediction error coefficients I.sub.err are equal to 0.

[0021]FIG. 4 shows the decoder 400 of the communication system.Bitstream 415 is received from encoder 300 of FIG. 3. Bitstream 415 isdemultiplexed via demultiplexer 410. Dequantized coefficientsd.sub.err(i,j) are calculated in the inverse quantization block 420:

d.sub.err(i, j)=Q ⁻¹(I.sub.err(i, j), QP).

[0022] In inverse transform block 430, the dequantized coefficients areused to obtain compressed prediction error by performing inversetransform:${E.{sub}.{c\left( {x,y} \right)}} = {\sum\limits_{i = 1}^{K}\quad {\sum\limits_{j = 1}^{L}{{d.{sub}.{{err}\left( {i,j} \right)}}{f.{sub}.i}\quad {{j\left( {x,y} \right)}.}}}}$

[0023] The pixels of the current coded frame are reconstructed byfinding the prediction pixels in the reference frame R(x,y) using thereceived motion vectors and then adding to the compressed predictionerror in adder 435:

I.sub.c(x, y)=R(x+Δx, y+Δ,y)+E.sub.c(x, y).

[0024] To obtain reconstructed image these values can be furthernormalized and filtered.

[0025] An example of a forward transform is provided by “H.26L TestModel Long Term Number 6 (TML-6) draft0”, document VCEG-L45, ITU-T VideoCoding Experts Group Meeting, Eibsee, Germany, 09-12 Jan. 2001. Theforward transformation of some pixels a, b, c, d into 4 transformcoefficients A, B, C, D is defined by:

A=13a+13b+13c+13d

B=17a+7b−7c−17d

C=13a−13b−13c+13d

D=7a−17b+17c−7d

[0026] The inverse transformation of transform coefficients A, B, C, Dinto 4 pixels a′, b′, c′, d′ is defined by:

a′=13A+17B+13C+7D

b′=13A+7B−13C−17D

c′=13A−7B−13C+17D

d′=13A−17B+13C−7D

[0027] The transform/inverse transform is performed for 4×4 blocks byperforming defined above one dimensional transform/inverse transformboth vertically and horizontally.

[0028] In “H.26L Test Model Long Term Number 6 (TML-6) draft0”, documentVCEG-L45, ITU-T Video Coding Experts Group Meeting, Eibsee, Germany,09-12 Jan. 2001 for chroma component, an additional 2×2 transform forthe DC coefficients is performed.

DCC(0,0)=(DC0+DC1+DC2+DC3)/2

DCC(1,0)=(DC0−DC1+DC2−DC3)/2

DCC(0,1)=(DC0+DC1−DC2−DC3)/2

DCC(1,1)=(DC0−DC1−DC2+DC3)/2

[0029] Definition of the corresponding inverse transform:

DC0=(DCC(0,0)+DCC(1,0)+DCC(0,1)+DCC(1,1))/2

DC1=(DCC(0,0)−DCC(1,0)+DCC(0,1)−DCC(1,1))/2

DC2=(DCC(0,0)+DCC(1,0)−DCC(0,1)−DCC(1,1))/2

DC3=(DCC(0,0)−DCC(1,0)−DCC(0,1)+DCC(1,1))/2

[0030] In “H.26L Test Model Long Term Number 6 (TML-6) draft0”, documentVCEG-L45, ITU-T Video Coding Experts Group Meeting, Eibsee, Germany,09-12 Jan. 2001 to obtain values of reconstructed image the results ofthe inverse transform are normalized by shifting by 20 bits (withrounding).

[0031] An example of quantization/dequantization is provided by “H.26LTest Model Long Term Number 6 (TML-6) draft0”, document VCEG-L45, ITU-TVideo Coding Experts Group Meeting, Eibsee, Germany, 09-12 Jan. 2001. Acoefficient c is quantized in the following way:

I=(c×A(QP)+f×2²⁰)//2²⁰

[0032] where f may be in the range (0-0.5) and f may have the same signas c. By // division with truncation is denoted. The dequantizedcoefficient is calculated as follows:

d=I×B(QP)

[0033] Values of A(QP) and B(QP) are given below:

[0034] A(QP=0, . . . , 31)=[620, 553, 492, 439, 391, 348, 310, 276, 246,219, 195, 174, 155, 138, 123, 110, 98, 87, 78, 69, 62, 55, 49, 44, 39,35, 31, 27, 24, 22, 19, 17];

[0035] B(QP=0, . . . , 31)=[3881, 4351, 4890, 5481, 6154, 6914, 7761,8718, 9781, 10987, 12339, 13828, 15523, 17435, 19561, 21873, 24552,27656, 30847, 34870, 38807, 437 47, 49103, 54683, 61694, 68745, 77615,89113, 100253, 109366, 126635, 141533];

[0036] Video streaming has emerged as one of the essential applicationsover the fixed internet and—in the near future over 3G multimedianetworks. In streaming applications, the server starts streaming thepre-encoded video bitstream to the receiver upon a request from thereceiver which plays the stream as it receives with a small delay. Theproblem with video streaming is that the best-effort nature of today'snetworks causes variations of the effective bandwidth available to auser due to the changing network conditions. The server should thenscale the bitrate of the compressed video to accommodate thesevariations. In case of conversational services that are characterized byreal-time encoding and point-to-point delivery, this is achieved byadjusting, on the fly, the source encoding parameters, such asquantization parameter or frame rate, based on the network feedback. Intypical streaming scenarios when already encoded video bitstream is tobe streamed to the client, the above solution can not be applied.

[0037] The simplest way of achieving bandwidth scalability in case ofpre-encoded sequences is by producing multiple and independent streamsof different bandwidth and quality. The server dynamically switchesbetween the streams to accommodate variations of the bandwidth availableto the client.

[0038] Now assume that we have multiple bitstreams generatedindependently with different encoding parameters, such as quantizationparameter, corresponding to the same video sequence. Since encodingparameters are different for each bitstream, the reconstructed frames ofdifferent bitstreams at the same time instant will not be the same.Therefore when switching between bitstreams, i.e., starting to decode abitstream, at arbitrary locations would lead to visual artifacts due tothe mismatch between the reference frames used to obtain predictedframe. Furthermore, the visual artifacts will not only be confined tothe switched frame but will further propagate in time due to motioncompensated coding.

[0039] In the current video encoding standards, perfect (mismatch-free)switching between bitstreams is achieved possible only at the positionswhere the future frames/regions do not use any information previous tothe current switching location, i.e., at I-frames. Furthermore, byplacing I-frames at fixed (e.g. 1 sec) intervals, VCR functionalities,such as random access or “Fast Forward” and “Fast Backward” (increasedplayback rate) for streaming video content, are achieved. User may skipa portion of video and restart playing at any I-frame location.Similarly, increased playback rate can be achieved by transmitting onlyI-pictures. The drawback of using I-frames in these applications is thatsince I-frames are not allowed to utilize temporal redundancy theyrequire much larger number of bits than P-frames.

[0040] The above-mentioned references are exemplary only and are notmeant to be limiting in respect to the resources and/or technologiesavailable to those skilled in the art.

SUMMARY

[0041] A new picture or frame type and method of using same is provided.This type of novel frame type is referred to as a SP-picture. SP-pictureuses motion compensated predictive coding to exploit temporal redundancyin the sequence. The difference between SP and P-pictures is that usingSP-pictures identical frames may be obtained even when differentreference frames are used for prediction. This property allowsSP-pictures to replace I-pictures in numerous applications such asswitching from one bitstream to another, random access, fast-forward,fast-backward. At the same time since SP-frames unlike I-frames areutilizing motion compensated predictive coding they require smallernumber of bits than I-frames.

[0042] These and other features, aspects, and advantages of embodimentsof the present invention will become apparent with reference to thefollowing description in conjunction with the accompanying drawings. Itis to be understood, however, that the drawings are designed solely forthe purposes of illustration and not as a definition of the limits ofthe invention, for which reference should be made to the appendedclaims.

BRIEF DESCRIPTIONS OF THE DRAWINGS

[0043]FIG. 1A is a diagram showing the encoding of an I-type frame orI-picture.

[0044]FIG. 1B is a diagram showing the encoding of a P-type frame orP-picture.

[0045]FIG. 1C is a diagram showing the encoding of a B-type frame orB-picture.

[0046]FIG. 2 is a diagram showing B-type frame inserted between I-typeand P-type frames and the direction which motion compensationinformation flows.

[0047]FIG. 3 is a block diagram of a generic motion-compensatedpredictive video coding system (encoder).

[0048]FIG. 4 is a block diagram of a generic motion-compensatedpredictive video coding system (decoder).

[0049]FIG. 5 is an illustration showing switching between bitstreams 1and 2 using SP-pictures.

[0050]FIG. 6 is a block diagram of a decoder in accordance with anembodiment of the invention.

[0051]FIG. 7 is an illustration of random access using SP-pictures.

[0052]FIG. 8 is an illustration of a fast-forward process usingSP-pictures.

DETAILED DESCRIPTION

[0053] A new decoder architecture is provided which has the propertythat identical frames may be obtained even when they are predicted usingdifferent reference frames. The picture type obtained using thisstructure will be called SP-frame also may be referred to as picture.This property allows SP-pictures to replace I-pictures in numerousapplications such as switching from one bitstream to another, randomaccess, fast-forward, fast-backward. Since unlike I-frames SP-frames areusing motion compensated prediction they require a lot less bits thanthe I-frames.

[0054] Some of possible applications of SP-frames are described below:

[0055] Bitstream Switching:

[0056] An example of how to utilize SP-frames to switch betweendifferent bitstreams is illustrated in the FIG. 5. FIG. 5 shows twobitstreams corresponding to the same sequence encoded at differentbitrates—bitstream 1 (510) and bitstream 2 (520). Within each encodedbitstream, SP-pictures should be placed at locations at which one wantsto allow switching from one bitstream to another (pictures S.sub.1(513), and S.sub.2 (523) in FIG. 5). When switching from bitstream 1(510) to bitstream 2 (520), another picture of this type will betransmitted (in FIG. 5 picture S.sub.12 (550) will be transmittedinstead of S.sub.2 (523)). Pictures S.sub.2 (523) and S.sub.12 (550) inFIG. 5 are represented by different bitstreams. However, theirreconstructed values are identical.

[0057] Random Access:

[0058] Application of SP-pictures to enable random access is depicted inFIG. 7. SP-pictures are placed at fixed intervals within bitstream 1(720) (e.g. picture S.sub.1 (730)) which is being streamed to theclient. To each one of these SP-pictures there is a corresponding pairof pictures generated and stored as another bitstream (bitstream 2(740)):

[0059] I-picture, I.sub.2 (750), at the temporal location precedingSP-picture.

[0060] SP-picture 710, S.sub.2, at the same temporal location asSP-picture.

[0061] Pictures stored in bitstream 2 (740) are only used when randomaccess is requested by a client. Bitstream 1 (720) may then be accessedat a location corresponding to an I-picture in bitstream 2 (740). Forexample to access bitstream 1 at frame I.sub.2, first the picturesI.sub.2, S.sub.2 from bitstream 2 are transmitted and then the followingpictures from bitstream 1 are transmitted.

[0062] Fast-Forward:

[0063] If in FIG. 7 bitstream 2 will constitute of only SP-picturespredicted from each other placed in larger temporal intervals (e.g. each1 sec) the structure presented in this figure can be used to obtain“Fast Forward” functionality. Due to the usage of SP-pictures “FastForward” can start at any bitstream location. In similar manner “FastBackward” functionality can be obtained.

[0064] Video Redundancy Coding:

[0065] SP-pictures have other uses in applications in which they do notact as replacements of I-pictures. Video Redundancy Coding can be givenas an example (VRC). “The principle of the VRC method is to divide thesequence of pictures into two or more threads in such a way that allcamera pictures are assigned to one of the threads in a round-robinfashion. Each thread is coded independently. In regular intervals, allthreads converge into a so-called sync frame. From this sync frame, anew thread series is started. If one of these threads is damaged becauseof a packet loss, the remaining threads stay intact and can be used topredict the next sync frame. It is possible to continue the decoding ofthe damaged thread, which leads to slight picture degradation, or tostop its decoding which leads to a drop of the frame rate. Sync framesare always predicted out of one of the undamaged threads. This meansthat the number of transmitted I-pictures can be kept small, becausethere is no need for complete re-synchronization.” For the sync frame,more than one representation (P-picture) is sent, each one using areference picture from a different thread. Due to the usage ofP-pictures these representations are not identical. Therefore, mismatchis introduced when some of the representations cannot be decoded andtheir counterparts are used when decoding the following threads. Usageof SP-pictures as sync frames eliminates this problem.

[0066] Error Resiliency/Recovery:

[0067] Multiple representations of a single frame in the form ofSP-frames predicted from different reference pictures, e.g., theimmediate previously reconstructed frames and a reconstructed framefurther back in time, can be used to increase error resilience. Considerthe case when an already encoded bitstream is being streamed and therehas been a packet loss leading to a frame loss. The client signals thelost frame(s) to the sender which responds by sending the next SP-framein the representation that uses frames that have been already receivedby the client.

[0068] In the following, we provide a detailed description of SP-pictureencoding/decoding within the context of H.26L.

[0069] SP-frame comprises blocks encoded using spatial correlation amongthe pixels (intra blocks) and blocks encoded using both spatial andtemporal correlation (inter or copy blocks). Inter and copy coded blocksare reconstructed differently than in P-type frames.

[0070] Value of each pixel S(x,y) in the inter or copy coded block isdecoded as a weighted sum of the basis functions f.sub.ij(x,y) where theweigh values d.sub.rec will be called dequantized reconstruction imagecoefficients. The values of d.sub.rec are obtained by quantization anddequantization of reconstruction image coefficients c.sub.rec.Reconstruction image coefficients c.sub.rec are formed using

[0071] Motion compensation prediction of this block constructed usingpreviously transmitted and decoded frames and received motion vectors.

[0072] Received quantized prediction error coefficients I.sub.err.

[0073] Values S(x,y) can be further normalized and filtered.

[0074] The invention is described in view of certain embodiments.Variations and modification are deemed to be within the spirit and scopeof the invention. The changes required in H.26L Test Model in order toimplement this embodiment of the present invention are also described.

[0075] SP-Picture Decoding

[0076] The decoding of inter and copy coded blocks in SP-picture isdescribed. Two different values of quantization parameter QP: QP1 andQP2 can be used during quantization/dequantization process for theseblocks. Values of QP1 and QP2 can be different whenquantization/dequantization process is performed for luma component thanwhen it is performed for chroma component.

[0077] The reconstructed values for inter and copy coded blocks arecalculated as follows:

[0078] 1. Form prediction P(x,y) of current block using received motionvectors and the reference frame. Calculate transform coefficientsc.sub.pred for P(x,y). These coefficients can be calculated byperforming forward transform on P(x,y).

[0079] 2. Calculate reconstruction image coefficients

c.sub.rec=c.sub.pred+alpha(QP2)×I.sub.err.

[0080]  where alpha(QP) is a parameter dependent on method ofquantization and used QP value. Quantize c.sub.rec using quantizationparameter QP=QP1. The quantized values will be referred to as quantizedreconstructed image coefficients and denoted as I.sub.rec.

[0081] When implementing this step in “H.26L Test Model Long Term Number6 (TML-6) draft0”, document VCEG-L45, ITU-T Video Coding Experts GroupMeeting, Eibsee, Germany, 09-12 Jan. 2001 to reduce computationalcomplexity calculation of c.sub.rec and their quantization are combined:

I.sub.rec=(c.sub.pred×A(QP1)+I.sub.err×F(QP1,QP2)+0.5×2²⁰)//2²⁰

[0082]  where F(QP1,QP2)=(2²⁰×A(QP1)+0.5×A(QP2)//A(QP2). Constant A(QP)is defined above in the section on quantization.

[0083] 3. Dequantize I.sub.rec using QP=QP1. The dequantizedcoefficients are equal to d.sub.rec.

[0084] 4. Inverse transform is performed for d.sub.rec. These values canbe further normalized and filtered.

[0085] Another Embodiment for SP-Picture Decoding

[0086] The blocks with type inter and copy are reconstructed as follows:

[0087] 1. Form prediction P(x,y) of current block using received motionvectors and the reference frame.

[0088] 2. Calculate transform coefficients c.sub.pred for P(x,y). Thesecoefficients can be calculated by performing forward transform forP(x,y).

[0089] 3. Quantize coefficients c.sub.pred using quantization parameterQP=QP1. The quantized values will be referred to as quantized predictionimage coefficients and denoted as I.sub.pred. Obtain quantizedreconstruction image coefficients I.sub.rec by adding the receivedquantized coefficients for the prediction error I.sub.err to I.sub.pred,i.e.,

I.sub.rec=I.sub.pred+(beta(QP2)×I.sub.err+0.5×beta(QP1))//beta(QP1).

[0090] where beta(QP) is a parameter dependent on method of quantizationand used QP value. In case of quantization used in “H.26L Test ModelLong Term Number 6 (TML-6) draft0”, document VCEG-L45, ITU-T VideoCoding Experts Group Meeting, Eibsee, Germany, 09-12 Jan. 2001,parameter beta(QP)=B(QP) where constant B(QP) is defined above in thesection on quantization.

[0091] 4. Dequantize I.sub.rec using QP=QP1. The dequantizedcoefficients are equal to d.sub.rec.

[0092] 5. Inverse transform is performed for d.sub.rec. These values canbe further normalized and filtered.

[0093] In the following, we describe the encoding of SP-frames for thedecoder structure described as the preferred embodiment of theinvention.

[0094] As can be observed from FIG. 5, there are two types of SP-frames,specifically, the SP-frames; placed within the bitstream, e.g., S.sub.1(513) and S.sub.2 (523) in FIG. 5, and the SP-frames (S.sub.12 in FIG.5) that will be sent when there is a switch between bitstreams (frombitstream 1 to bitstream 2). The encoding of S.sub.2 (523) and S.sub.12(550) are such that their reconstructed frames are identical althoughthey use different reference frames as described below.

[0095] When encoding an SP-picture placed within a bitstream (S.sub.1(513) and S.sub.2 (523) in FIG. 5), the encoding of inter and copy codedblocks is performed as follows:

[0096] 1. Calculate motion vectors using same method as for P-pictures.After motion compensation calculate transform coefficients for predictedP(x,y) and current image I(x,y) by performing forward transform. Thetransform coefficients for the current image are denoted as c.sub.origand for the predicted image as c.sub.pred.

[0097] 2. Transform coefficients for the predicted blocks are quantizedusing QP=QP1. Obtained levels are denoted as I.sub.pred.

[0098] 3. The prediction error coefficients are obtained byc.sub.err=c.sub.orig−I.sub.pred×alpha(QP1) where alpha(QP) is aparameter dependent on method of quantization and used QP value.

[0099] When SP-frames are used in “H.26L Test Model Long Term Number 6(TML-6) draft0”, document VCEG-L45, ITU-T Video Coding Experts GroupMeeting, Eibsee, Germany, 09-12 Jan. 2001

alpha(QP)=(2²⁰+0.5×A(QP))//A(QP)

[0100]  where constant A(QP) is defined above in the section onquantization.

[0101] 4. The prediction error coefficients are quantized using QP=QP2.It is recommended to use rate distortion constrained quantization (RDQ)in which more than one coefficient are considered together in thequantization process. An example of use rate distortion constrainedquantization is described in “H.26L Test Model Long Term Number 6(TML-6) draft0”, document VCEG-L45, ITU-T Video Coding Experts GroupMeeting, Eibsee, Germany, 09-12 Jan. 2001.

[0102] Let as assume that we want to encode the SP-picture, denoted asS.sub.12 in FIG. 5, to switch from bitstream 1 to bitstream 2. Thereconstructed values of this picture have to be identical to thereconstructed values of SP-picture in bitstream. 2, denoted as S.sub.2in FIG. 5, to which we are switching. The bitstream of the Intramacroblocks in frame S.sub.2 are copied to S.sub.12. The encoding ofinter macroblocks is performed as follows:

[0103] 1. Form the predicted frame for S.sub.12 by performing motionestimation with the reference frames being pictures preceding S.sub.1 inbitstream 1.

[0104] 2. Calculate transform coefficients for predicted image byperforming forward transform. The transform coefficients for thepredicted image are denoted as c.sub.pred.

[0105] 3. Quantize the obtained coefficients c.sub.pred using QP=QP1 andsubtract the quantized coefficient levels I.sub.pred from thecorresponding I.sub.rec of S.sub.2-picture. The resulting levels are thelevels of the prediction error which will be transmitted to the decoder.

[0106] An embodiment of a decoder 600 in accordance with an embodimentof the invention is illustrated in FIG. 6. Referring to FIG. 6, dedcoder600 comprises, inter alia, a demultiplexer 610, inverse quantizationblock 620, m inverse transform block 630, frame memory 640, MCprediction block 650, tranform block 660, quantization block 670.

[0107] The invention is described in view of certain embodiments.Variations and modification are deemed to be within the spirit and scopeof the invention. For instance, data from the demultiplexer may benormalized before proceeding to adder and inverse quantization. It willbe obvious to those skilled in the art after reading the specificationincluding the appended claims that various changes in form and detailmay be made therein without departing from the spirit and scope of theinvention.

1-13. (cancelled)
 14. A method for providing Video Redundancy Coding(VRC), comprising the steps of: dividing a sequence of pictures into aplurality of threads wherein all pictures are assigned to one of saidplurality of threads in a round-robin fashion; coding each of saidplurality of threads independently; creating a SP-picture, wherein allof said threads converge; and starting a second plurality of threadsfrom said SP-picture.
 15. A method for providing error control in a datastream between a sender and a client in a communication system, saidmethod comprising: creating a plurality of representations of a picturein the form of a plurality of SP-pictures predicted from differentreference pictures; signaling said sender information regarding lostpictures and a one of said plurality of representations received by saidclient; and sending said client a SP-picture which is the next picturein said one of plurality of representations received by client.
 16. Apicture data signal embodied in a carrier wave, comprising: data blocksencoded using spatial correlation among the pixels (intra blocks); anddata blocks encoded using both spatial and temporal correlation (interor copy blocks), wherein the Inter and copy coded blocks arereconstructed differently than in P-type pictures.
 17. A picture datastructure, comprising: data blocks encoded using spatial correlationamong the pixels (intra blocks); and data blocks encoded using bothspatial and temporal correlation (inter or copy blocks), wherein theInter and copy coded blocks are reconstructed differently than in P-typepictures.
 18. The picture data structure of claim 17 wherein the valueof each pixel S(x,y) in the inter or copy coded block is decoded as aweighted sum of a plurality of basis functions whose weigh values areobtained by quantization and dequantization of a plurality ofreconstruction image coefficients formed using motion compensationprediction of this block constructed using previously transmitted anddecoded pictures, received motion vectors and received quantizedprediction error coefficients. 19-22. (cancelled)